Design Space Exploration of DNNs for Autonomous Systems

Indiana University-Purdue University Indianapolis (IUPUI)Developing intelligent agents that can perceive and understand the rich visualworld around us has been a long-standing goal in the field of AI. Recently, asignificant progress has been made by the CNNs/DNNs to the incredible advances& in a wide range of applications such as ADAS, intelligent cameras surveillance,autonomous systems, drones, & robots. Design space exploration (DSE) of NNs andother techniques have made CNN/DNN memory & computationally efficient. Butthe major design hurdles for deployment are limited resources such as computation,memory, energy efficiency, and power budget. DSE of small DNN architectures forADAS emerged with better and efficient architectures such as baseline SqueezeNetand SqueezeNext. These architectures are exclusively known for their small modelsize, good model speed & model accuracy.In this thesis study, two new DNN architectures are proposed. Before diving intothe proposed architectures, DSE of DNNs explores the methods to improveDNNs/CNNs.Further, understanding the different hyperparameters tuning &experimenting with various optimizers and newly introduced methodologies. First,High Performance SqueezeNext architecture ameliorate the performance of existingDNN architectures. The intuition behind this proposed architecture is to supplantconvolution layers with a more sophisticated block module & to develop a compactand efficient architecture with a competitive accuracy. Second, Shallow SqueezeNextarchitecture is proposed which achieves better model size results in comparison tobaseline SqueezeNet and SqueezeNext is presented. It illustrates the architecture is xviicompact, efficient and flexible in terms of model size and accuracy.Thestate-of-the-art SqueezeNext baseline and SqueezeNext baseline are used as thefoundation to recreate and propose the both DNN architectures in this study. Dueto very small model size with competitive model accuracy and decent model testingspeed it is expected to perform well on the ADAS systems.The proposedarchitectures are trained and tested from scratch on CIFAR-10 [30] & CIFAR-100[34] datasets. All the training and testing results are visualized with live loss andaccuracy graphs by using livelossplot. In the last, both of the proposed DNNarchitectures are deployed on BlueBox2.0 by NXP

Shallow SqueezeNext: An Efficient & Shallow DNN

CNN has gained great success in many applications but the major design hurdles for deploying CNN on driver assistance systems or ADAS are limited computation, memory resource, and power budget. Recently, there has been greater exploration into small DNN architectures, such as SqueezeNet and SqueezeNext architectures. In this paper, the proposed Shallow SqueezeNext architecture for driver assistance systems achieves better model size with a good model accuracy and speed in comparison to baseline SqueezeNet and SqueezeNext architectures. The proposed architecture is compact, efficient and flexible in terms of model size and accuracy with minimum tradeoffs and less penalty. The proposed Shallow SqueezeNext uses SqueezeNext architecture as its motivation and foundation. The proposed architecture is developed with intention for implementation or deployment on a real-time autonomous system platform and to keep the model size less than 5 MB. Due to its extremely small model size, 0.370 MB with a competitive model accuracy of 82.44 %, decent both training and testing model speed of 7 seconds, it can be successfully deployed on ADAS, driver assistance systems or a real time autonomous system platform such as BlueBox2.0 by NXP. The proposed Shallow SqueezeNext architecture is trained and tested from scratch on CIFAR-10 dataset for developing a dataset specific trained model

High Performance SqueezeNext for CIFAR-10

CNNs is the foundation for deep learning and computer vision domain enabling applications such as autonomous driving, face recognition, automatic radiology image reading, etc. But, CNN is a algorithm which is memory and computationally intensive. DSE of neural networks and compression techniques have made convolution neural networks memory and computationally efficient. It improved the CNN architectures and made it more suitable to implement on real-time embedded systems. This paper proposes an efficient and a compact CNN to ameliorate the performance of existing CNN architectures. The intuition behind this proposed architecture is to supplant convolution layers with a more sophisticated block module and to develop a compact architecture with a competitive accuracy. Further, explores the bottleneck module and squeezenext basic block structure. The state-of-the-art squeezenext baseline architecture is used as a foundation to recreate and propose a high performance squeezenext architecture. The proposed architecture is further trained on the CIFAR-10 dataset from scratch. All the training and testing results are visualized with live loss and accuracy graphs. Focus of this paper is to make an adaptable and a flexible model for efficient CNN performance which can perform better with the minimum tradeoff between model accuracy, size, and speed. Finally, the conclusion is made that the performance of CNN can be improved by developing an architecture for a specific dataset. The purpose of this paper is to introduce and propose high performance squeezenext for CIFAR-10

Shallow SqueezeNext: Real Time Deployment on Bluebox2.0 with 272KB Model Size

The significant challenges for deploying CNNs/DNNs on ADAS are limited computation and memory resources with very limited efficiency. Design space exploration of CNNs or DNNS, training and testing DNN from scratch, hyper parameter tuning, implementation with different optimizers contributed towards the efficiency and performance improvement of the Shallow SqueezeNext architecture. It is also computationally efficient, inexpensive and requires minimum memory resources. It achieves better model size and speed in comparison to other counterparts such as AlexNet, VGGnet, SqueezeNet, and SqueezeNext, trained and tested from scratch on datasets such as CIFAR-10 and CIFAR-100. It can achieve the least model size of 272KB with a model accuracy of 82%, a model speed of 9 seconds per epoch, and tested on the CIFAR-10 dataset. It achieved the best accuracy of 91.41%, best model size of 0.272 MB, and best model speed of 4 seconds per epoch. Memory resources are of high importance when it comes down to real time system or platforms because usually the memory is quite limited. To verify that the Shallow SqueezeNext can be successfully deployed on a real time platform, bluebox2.0 by NXP was used. Bluebox2.0 deployment of Shallow SqueezeNext architecture achieved a model accuracy of 90.50%, 8.72MB model size and 22 seconds per epoch model speed. There is another version of the Shallow SqueezeNext which performed better that attained a model size of 0.5MB with model accuracy of 87.30% and 11 seconds per epoch model speed trained and tested from scratch on CIFAR-10 dataset